Multi-lingual audio streaming

ABSTRACT

Digital video data and digital multiple-audio data are extracted from a source, using a hardware processor in a content source device within a premises. The extracted digital video data is processed for display on a main display device in the premises; and the extracted digital multiple-audio data is processed into a primary soundtrack in a primary language, to be listened to within the premises in synchronization with the displayed extracted digital video data. The primary soundtrack corresponds to the displayed extracted digital video data, in the primary language. The extracted digital multiple-audio data is processed into at least one secondary audio asset, different than the primary soundtrack; and the at least one secondary audio asset is transmitted to a personal media device within the premises, for apprehension by a user of the personal media device in synchronization with the displayed extracted digital video data.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation, under 37 CFR 1.53(b), of co-assignedU.S. patent application Ser. No. 13/094,992 of inventors SherisseHawkins et al., and claims the benefit thereof, said application Ser.No. 13/094,992 having been filed on Apr. 27, 2011, and entitled“MULTI-LINGUAL AUDIO STREAMING.” The complete disclosure of theaforesaid application Ser. No. 13/094,992 is expressly incorporatedherein by reference in its entirety for all purposes.

FIELD OF THE INVENTION

The present invention relates generally to the electrical, electronic,and computer arts, and more particularly relates to video contentnetworks.

BACKGROUND OF THE INVENTION

With the advent of digital communications technology, many TV programstreams are transmitted in digital formats. For example, DigitalSatellite System (DSS), Digital Broadcast Services (DBS), and AdvancedTelevision Standards Committee (ATSC) program streams are digitallyformatted pursuant to the well-known Moving Pictures Experts Group 2(MPEG-2) standard. The MPEG-2 standard specifies, among other things,the methodologies for video and audio data compression allowing formultiple programs, with different video and audio feeds, to bemultiplexed in a transport stream traversing a single transmissionchannel. A digital TV receiver may be used to decode an MPEG-2 encodedtransport stream, and extract the desired program therefrom.

The compressed video and audio data are typically carried by continuouselementary streams, respectively, which are broken into access units orpackets, resulting in packetized elementary streams (PESs). Thesepackets are identified by headers that contain time stamps forsynchronizing, and are used to form MPEG-2 transport streams. Fordigital broadcasting, multiple programs and their associated PESs aremultiplexed into a single transport stream. A transport stream has PESpackets further subdivided into short fixed-size data packets, in whichmultiple programs, encoded with different clocks, can be carried. Atransport stream not only includes a multiplex of audio and video PESs,but also other data such as MPEG-2 program specific information(sometimes referred to as metadata) describing the transport stream. TheMPEG-2 metadata may include a program associated table (PAT) that listsevery program in the transport stream. Each entry in the PAT points toan individual program map table (PMT) that lists the elementary streamsmaking up each program. Some programs are open, but some programs may besubject to conditional access (encryption), and this information (i.e.,whether open or subject to conditional access) is also carried in theMPEG-2 transport stream, typically as metadata.

The aforementioned fixed-size data packets in a transport stream eachcarry a packet identifier (PID) code. Packets in the same elementarystreams all have the same PID, so that a decoder can select theelementary stream(s) it needs and reject the remainder.Packet-continuity counters may be implemented to ensure that everypacket that is needed to decode a stream is received.

Within recent years a proliferation of hand held multi-media deviceshave entered the market. These devices contain visual displays as wellas audio outputs and connectivity to wireless Internet and intranet via,for example, IEEE 802.11n or Bluetooth® technology (registeredcertification mark of BLUETOOTH SIG, INC. Suite 350, 5209 LakeWashington Boulevard, Kirkland Wash. 98033, USA). Many of these devicesprovide methods for software developers who are not related to themanufacturer to embellish their capabilities with their ownapplications. Examples of such manufacturers are Apple Inc. and GoogleInc. Apple provides a device called the IPOD TOUCH® (registered mark ofApple Inc., Cupertino Calif. 95014 USA) and Google provides a multipleplatform operating system called ANDROID® (registered mark of GoogleInc., 1600 Amphitheatre Parkway, Mountain View Calif. 94043 USA) thatenables similar functionality on a plethora of devices. Cable televisionand Telecom Internet providers have provided household connectivity byboth wired and wireless systems, such that television content can bepassed between multiple internet-connected devices, which present videoand audio content to their customers.

At this point, it should be noted that market research has shown that anincreasing number of families have a mixture of language capability. Asfamilies migrate to the United States the younger generation learnsEnglish while the older generations remain speaking their nativelanguage. In this regard, it is known (so-called subtitles) to provide atextual representation of the words spoken in one language that aretranslated into another and overlaid on the television screen. DigitalVersatile Disks (DVDs) may contain a selection of languages that can bepresented, but only one language at a time can be played. Furthermore,museums and special exhibits have also provided multi-lingual audiodevices to provide individually paced audio tours.

SUMMARY OF THE INVENTION

Techniques are provided for multi-lingual audio streaming.

In one aspect, an exemplary method includes the steps of extractingdigital video data and digital multiple-audio data from a source, usinga hardware processor in a content source device within a premises;processing the extracted digital video data for display on a maindisplay device in the premises; and processing the extracted digitalmultiple-audio data into a primary soundtrack in a primary language, tobe listened to within the premises in synchronization with the displayedextracted digital video data. The primary soundtrack corresponds to thedisplayed extracted digital video data, in the primary language.Additional steps include processing the extracted digital multiple-audiodata into at least one secondary audio asset, different than the primarysoundtrack; and transmitting the at least one secondary audio asset to apersonal media device within the premises, for apprehension by a user ofthe personal media device in synchronization with the displayedextracted digital video data.

In another aspect, an exemplary method is provided for interactionbetween a personal media device and a content source device whichprovides, within a premises, digital video data for display on a maindisplay device in the premises, and a primary soundtrack in a primarylanguage, to be listened to within the premises in synchronization withthe displayed digital video data. The method includes the step ofobtaining, at the personal media device within the premises, from thecontent source device within the premises, a representation of a menu.The representation of the menu includes data indicative of availableforms of a secondary audio asset. Additional steps include providing,from the personal media device, to the content source device, a menuselection specifying a chosen form of the secondary audio asset; andobtaining, at the personal media device, from the content source device,the secondary audio asset, for apprehension by a user of the personalmedia device in synchronization with the displayed digital video data.The secondary audio asset is different than the primary soundtrack.

In still another aspect, an exemplary system includes a main memory; amain hardware processor coupled to the main memory; a source of digitalvideo data and digital multiple-audio data in data communication withthe main hardware processor; a wireless interface coupled to the mainhardware processor; and a graphics processor coupled to the mainhardware processor. The main hardware processor and the main memory arecooperatively configured to: extract the digital video data and thedigital multiple-audio data from the source; process the extracteddigital video data and route the extracted video data to the graphicsprocessor for display in a premises; and process the extracted digitalmultiple-audio data into a primary soundtrack in a primary language, tobe listened to within the premises in synchronization with the displayedextracted digital video data. The primary soundtrack corresponds to thedisplayed extracted digital video data, in the primary language. Themain hardware processor and the main memory are also cooperativelyconfigured to process the extracted digital multiple-audio data into atleast one secondary audio asset, different than the primary soundtrack.The wireless interface is configured to facilitate transmitting the atleast one secondary audio asset to a personal media device within thepremises, for apprehension by a user of the personal media device insynchronization with the displayed extracted digital video data.

In some instances, the system further includes one or more appropriatelyconfigured personal media devices.

As used herein, “facilitating” an action includes performing the action,making the action easier, helping to carry the action out, or causingthe action to be performed. Thus, by way of example and not limitation,instructions executing on one processor might facilitate an actioncarried out by instructions executing on a remote processor, by sendingappropriate data or commands to cause or aid the action to be performed.For the avoidance of doubt, where an actor facilitates an action byother than performing the action, the action is nevertheless performedby some entity or combination of entities.

One or more embodiments of the invention or elements thereof can beimplemented in the form of an article of manufacture including a machinereadable medium that contains one or more programs which when executedimplement such step(s); that is to say, a computer program productincluding a tangible computer readable recordable storage medium (ormultiple such media) with computer usable program code for performingthe method steps indicated. Furthermore, one or more embodiments of theinvention or elements thereof can be implemented in the form of anapparatus including a memory and at least one processor that is coupledto the memory and operative to perform, or facilitate performance of,exemplary method steps. Yet further, in another aspect, one or moreembodiments of the invention or elements thereof can be implemented inthe form of means for carrying out one or more of the method stepsdescribed herein; the means can include (i) specialized hardwaremodule(s), (ii) software module(s) stored in a tangiblecomputer-readable recordable storage medium (or multiple such media) andimplemented on a hardware processor, or (iii) a combination of (i) and(ii); any of (i)-(iii) implement the specific techniques set forthherein.

One or more embodiments of the invention can provide substantialbeneficial technical effects, including bandwidth savings within apremises environment by selectively broadcasting only needed secondaryaudio assets.

These and other features and advantages of the invention will becomeapparent from the following detailed description of illustrativeembodiments thereof, which is to be read in connection with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional block diagram illustrating an exemplary hybridfiber-coaxial (HFC) network configuration;

FIG. 1A is a functional block diagram illustrating one exemplary HFCcable network head-end configuration;

FIG. 1B is a functional block diagram illustrating one exemplary localservice node configuration;

FIG. 2 shows at least a portion of a system for multi-lingual audiostreaming within a home or other premises, according to an aspect of theinvention;

FIG. 3 is a functional block diagram of a content source device of thesystem of FIG. 2, according to an aspect of the invention;

FIG. 4 shows exemplary software components within a typical contentsource and their relationship to each other within the content source,according to an aspect of the invention;

FIG. 5 shows exemplary software components within a personal mediadevice (PMD) application, according to an aspect of the invention;

FIG. 6 shows the hardware within a typical PMD that is used by one ormore embodiments of the invention;

FIG. 7 is a flow chart showing exemplary method steps from theperspective of the content source device, according to an aspect of theinvention;

FIG. 8 is a flow chart showing exemplary method steps from theperspective of the personal media device, according to an aspect of theinvention; and

FIG. 9 is a block diagram of an exemplary computer system useful inimplementing at least a portion of one or more embodiments of theinvention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

FIG. 1 illustrates a typical content-based network configuration 100.The various components of the network 100 include (i) one or more dataand application origination points 102; (ii) one or more content sources103, (iii) one or more application distribution servers 104; (iv) one ormore video-on-demand (VOD) servers 105, and (v) consumer (or customer)premises equipment (CPE) 106. Also included is a dynamic bandwidthallocation device (DBWAD) 1001 such as a global session resourcemanager, which is itself a non-limiting example of a session resourcemanager. The distribution server(s) 104, VOD servers 105, DBWAD 1001,and CPE(s) 106 are connected via a bearer (e.g., hybrid fiber cable(HFC)) network 101. A simple architecture is shown in FIG. 1 forillustrative brevity, although it will be recognized that comparablearchitectures with multiple origination points, distribution servers,VOD servers, and/or CPE devices (as well as different networktopologies) may be utilized consistent with the invention. For example,the head-end architecture of FIG. 1A (described in greater detail below)may be used.

It should be noted at this point that in addition to a conventional HFCnetwork or a switched digital network, other kinds of video contentnetworks can be employed for network 101 (e.g., fiber-to-the-home (FTTH)or fiber-to-the-curb (FTTC)).

The data/application origination point 102 comprises any medium thatallows data and/or applications (such as a VOD-based or “Watch TV”application) to be transferred to a distribution server 104 (forexample, over a suitable network, not separately numbered). This caninclude for example a third party data source, application vendorwebsite, compact disk read-only memory (CD-ROM), external networkinterface, mass storage device (e.g., Redundant Arrays of InexpensiveDisks (RAID) system), etc. Such transference may be automatic, initiatedupon the occurrence of one or more specified events (such as the receiptof a request packet or acknowledgement (ACK)), performed manually, oraccomplished in any number of other modes readily recognized by those ofordinary skill.

The application distribution server 104 comprises a computer systemwhere such applications can enter the network system. Distributionservers per se are well known in the networking arts.

The VOD server 105 comprises a computer system where on-demand contentcan be received from one or more of the aforementioned data sources 102and enter the network system. These servers may generate the contentlocally, or alternatively act as a gateway or intermediary from adistant source.

The CPE 106 includes any equipment in the customers' premises (or otherappropriate locations) that can be accessed by a distribution server104; for example, set-top terminal (STT), digital set-top box (DSTB),set-top box (STB), or simply “box,” and the like.

Referring now to FIG. 1A, one exemplary embodiment of a head-endarchitecture is described. As shown in FIG. 1A, the head-endarchitecture 150 comprises typical head-end components and servicesincluding billing module 152, subscriber management system (SMS) and CPEconfiguration management module 308, cable-modem termination system(CMTS) and out-of-band (OOB) system 156, as well as LAN(s) 158, 160placing the various components in data communication with one another.It will be appreciated that while a bar or bus LAN topology isillustrated, any number of other arrangements (e.g., ring, star, etc.)may be used consistent with the invention. It will also be appreciatedthat the head-end configuration depicted in FIG. 1A is high-level,conceptual architecture and that each multi-service operator or multiplesystem operator (MSO) may have multiple head-ends deployed using customarchitectures.

The architecture 150 of FIG. 1A further includes amultiplexer/encrypter/modulator (MEM) 162 coupled to the HFC network 101adapted to “condition” content for transmission over the network. Thedistribution servers 104 are coupled to the LAN 160, which providesaccess to the MEM 162 and network 101 via one or more file servers 170.The VOD servers 105 are coupled to the LAN 158, although otherarchitectures may be employed (such as for example where the VOD serversare associated with a core switching device such as an 802.3z GigabitEthernet device; or the VOD servers could be coupled to LAN 160). Sinceinformation is typically carried across multiple channels, the head-endshould be adapted to acquire the information for the carried channelsfrom various sources. Typically, the channels being delivered from thehead-end 150 to the CPE 106 (“downstream”) are multiplexed together inthe head-end and sent to neighborhood hubs (see FIG. 1B) via a varietyof interposed network components.

Content (e.g., audio, video, etc.) is provided in each downstream(in-band) channel associated with the relevant service group. Tocommunicate with the head-end or intermediary node (e.g., hub server),the CPE 106 may use the out-of-band (OOB) or DOCSIS® (Data Over CableService Interface Specification) channels (registered mark of CableTelevision Laboratories, Inc., 400 Centennial Parkway Louisville Colo.80027, USA) and associated protocols. The OpenCable™ ApplicationPlatform (OCAP) 1.0, 2.0, 3.0 (and subsequent) specification (CableTelevision laboratories Inc.) provides for exemplary networkingprotocols both downstream and upstream, although the invention is in noway limited to these approaches. All versions of the DOCSIS and OCAPspecifications are expressly incorporated herein by reference in theirentireties for all purposes.

It will also be recognized that multiple servers (broadcast, VOD, orotherwise) can be used, and disposed at two or more different locationsif desired, such as being part of different server “farms”. Thesemultiple servers can be used to feed one service group, or alternativelydifferent service groups. In a simple architecture, a single server isused to feed one or more service groups. In another variant, multipleservers located at the same location are used to feed one or moreservice groups. In yet another variant, multiple servers disposed atdifferent location are used to feed one or more service groups.

In some instances, material may also be obtained from a satellite feed1108; such material is demodulated and decrypted in block 1106 and fedto block 162. Conditional access system 157 may be provided for accesscontrol purposes. Network management system 1110 may provide appropriatemanagement functions. Note also that signals from MEM 162 and upstreamsignals from network 101 that have been demodulated and split in block1112 are fed to CMTS and 00B system 156.

Also included in FIG. 1A are a global session resource manager (GSRM)302, a Mystro Application Server 104A, and a business management system154, all of which are coupled to LAN 158. GSRM 302 is one specific formof a DBWAD 1001 and is a non-limiting example of a session resourcemanager.

As shown in FIG. 1B, the network 101 of FIGS. 1 and 1A comprises afiber/coax arrangement wherein the output of the MEM 162 of FIG. 1A istransferred to the optical domain (such as via an optical transceiver177 at the head-end 150 or further downstream). The optical domainsignals are then distributed to a fiber node 178, which furtherdistributes the signals over a distribution network 180 to a pluralityof local servicing nodes 182. This provides an effective 1:N expansionof the network at the local service end.

US Patent Publication 2003-0056217 of Paul D. Brooks, entitled“Technique for Effectively Providing Program Material in a CableTelevision System,” the complete disclosure of which is expresslyincorporated herein by reference for all purposes, describes oneexemplary broadcast switched digital architecture useful with one ormore embodiments of the present invention, although it will berecognized by those of ordinary skill that other approaches andarchitectures may be substituted. In a cable television system inaccordance with the Brooks invention, program materials are madeavailable to subscribers in a neighborhood on an as needed basis.Specifically, when a subscriber at a set-top terminal selects a programchannel to watch, the selection request is transmitted to a head end ofthe system. In response to such a request, a controller in the head enddetermines whether the material of the selected program channel has beenmade available to the neighborhood. If it has been made available, thecontroller identifies to the set-top terminal the carrier which iscarrying the requested program material, and to which the set-topterminal tunes to obtain the requested program material. Otherwise, thecontroller assigns an unused carrier to carry the requested programmaterial, and informs the set-top terminal of the identity of the newlyassigned carrier. The controller also retires those carriers assignedfor the program channels which are no longer watched by the subscribersin the neighborhood.

Note that reference is made herein, for brevity, to features of the“Brooks invention”—it should be understood that no inference should bedrawn that such features are necessarily present in all claimedembodiments of Brooks.

The Brooks invention is directed to a technique for utilizing limitednetwork bandwidth to distribute program materials to subscribers in acommunity access television (CATV) system. In accordance with the Brooksinvention, the CATV system makes available to subscribers selectedprogram channels, as opposed to all of the program channels furnished bythe system as in prior art. In the Brooks CATV system, the programchannels are provided on an as needed basis, and are selected to servethe subscribers in the same neighborhood requesting those channels.

US Patent Publication 2010-0313236 of Albert Straub, entitled“TECHNIQUES FOR UPGRADING SOFTWARE IN A VIDEO CONTENT NETWORK,” thecomplete disclosure of which is expressly incorporated herein byreference for all purposes, provides additional details on theaforementioned dynamic bandwidth allocation device.

US Patent Publication 2009-0248794 of William L. Helms, entitled “SYSTEMAND METHOD FOR CONTENT SHARING,” the complete disclosure of which isexpressly incorporated herein by reference for all purposes, providesadditional details on CPE in the form of a converged premises gatewaydevice.

In one or more embodiments, an exemplary method provides the audioportion of an audio-video presentation in a plurality of languagessimultaneously to multiple users and an exemplary system distributesindividual channels of the audio to specific users under control of theusers.

As noted above, market research has shown that an increasing number offamilies have a mixture of language capability. As families migrate tothe United States the younger generation learns English while the oldergenerations remain speaking their native language. In this regard, it isknown (so-called subtitles) to provide a textual representation of thewords spoken in one language that are translated into another andoverlaid on the television screen. Text on the video overlays portionsof the screen presentation and may obscure visual items making suchtechnology less than desirable. Digital Versatile Disks (DVDs) maycontain a selection of languages that can be presented, but only onelanguage at a time can be played. Furthermore, museums and specialexhibits have also provided multi-lingual audio devices to provideindividually paced audio tours. These devices and systems suffer from alack of synchronization between listeners.

As also noted above, within recent years a proliferation of hand heldmulti-media devices have entered the market. These devices containvisual displays as well as audio outputs and connectivity to wirelessInternet and Intranet via, for example, IEEE 802.11n or Bluetooth®technology. Many of these devices provide methods for softwaredevelopers who are not related to the manufacturer to embellish theircapabilities with their own applications. Examples of such manufacturersare Apple Inc. and Google Inc. Apple provides a device called the IPODTOUCH® and Google provides a multiple platform operating system calledANDROID® that enables similar functionality on a plethora of devices.Cable television and Telecom Internet providers have provided householdconnectivity by both wired and wireless systems, such that televisioncontent can be passed between multiple internet-connected devices, whichpresent video and audio content to their customers.

One or more embodiments of the invention provide per-user customizablelanguage audio options to groups of individuals sharing the same videopresentation.

One or more embodiments of the invention provide per-user customizableaccess to alternative audio content options to groups of individualssharing the same video presentation.

One or more embodiments of the invention provide per-user customizableaccess to language-specific textual displays of the audio content togroups of individuals sharing the same video presentation.

One or more embodiments of the invention provide per-user customizableaccess to language-specific ancillary information concerning a videodisplay being shared between members of a group of observers.

Some embodiments provide, as a secondary audio asset, “DescriptiveAudio” as defined by the “Twenty-First Century Communications and VideoAccessibility Act of 2010.” This is not a language per se but is anadditional audible track. This descriptive audio is typically used byvisually impaired persons who wish to enjoy television, and typicallyincludes descriptions of what is happening in the video as well as thenormal dialog and other sounds, music, and so on. In general, it may bein the same or a different language as the primary soundtrack in theprimary language.

In at least some instances, a software application on portablemulti-media devices communicates with the home wireless system andallows the user to (i) open a data channel to the device providing thecontent; (ii) select one of a plurality of available audio ordescriptive text options available with the content; and (iii) beginlistening to the audio selection or reading the text that issynchronized with the visual display.

Initially, an overview of FIGS. 2-6 will be provided. Referring now toFIG. 2, elements 901 is a source of digital audio and video content thatcontains a mechanism for transmitting the audio and textual descriptivecontent to a remote presentation device. Element 902 is a video displaydevice such as a television or video monitor that may embody within itthe digital content source shown as 901. Element 903 is a personal mediadevice such as an Apple IPOD TOUCH® or a high end cell phone with apersonal listening device such as a headphone attached. Element 904 is aconnection between the audio/video source device and the intranet and/orInternet; this connection may be wireless as well. Symbols 905 representa radio frequency communication channel that is accessible by thepersonal media devices and the audio/video source.

Referring to FIG. 3, a preferred embodiment of a content source device901 includes a processor 201; memory 202; disk drive 203 (or otherpersistent memory device); a wireless connection or a wired connectionwith access to a wireless router, designated generally as 204; and a DVDor other removable digital storage device 205. Also included are audioprocessor 206, graphics processor 207, and a programmable variable delay208 (such as a FIFO buffer).

FIG. 4 shows exemplary software components within a typical contentsource 901 and their relationship to each other within the contentsource. Element 301 is the actual video, audio, text, and descriptivedata; element 302 is a Digital Living Network Alliance (DLNA) or similardata presentation and/or discovery component; and element 303 is thesoftware driver responsible for streaming the content to the video andaudio devices (e.g., an internet protocol (IP) stack or other channel).

FIG. 5 depicts exemplary software components that make up the personalmedia device application. Element 401 is the user interface used toselect the content desired as well as to configure the device anddisplay text; element 402 is the audio device driver that sends theaudio data to the audio hardware; element 403 is a DLNA protocol stackor equivalent mechanism to allow for the discovery and presentation ofthe content available on the content source; and element 404 is anetwork device driver (e.g., internet protocol (IP) stack) that sendsand receives the DLNA protocol information to the content source as wellas receives the text and audio data to be shown or played.

FIG. 6 depicts hardware within a typical PMD that is used by one or moreembodiments of the invention. Element 501 is a processor used to executethe application; and element 502 is a combination of volatile andnon-volatile memory (e.g., random access memory (RAM) and electronicallyerasable programmable read-only memory (EEPROM)). The executableapplication program, operating system, and the like are stored in thenon-volatile memory and the Random Access Memory is used for programexecution variable storage as well as to buffer the audio and displaydata. Audio processor 503 is a device that converts digital audio datainto the electrical signals which drive the audio transducers (speakers)within headphones to generate sound. Element 504 is a graphics processorthat is connected to a small display device (not separately shown) andwhich provides the visual interface to the user. Element 505 is anetwork interface controller (NIC) that connects wirelessly to thecontent source 901 via the home Intranet or Internet. Element 506 is akeyboard containing a set of keys for selecting the language andadjusting the synchronization delay if necessary.

FIG. 2 shows a preferred embodiment. The content source device ispreferably a television set top box 901 (“STB” or “CPE”) connected to avideo display 902. This preference is due to the rapid rate of change ofset-top box technology compared to the relatively slow rate of change ofthe display technology. When the components are combined, the user runsthe risk of having an obsolete display device simply because the contentsource—DVD, Internet, Cable, and the like became obsolete. Content ispreferably stored in the MPEG-4 (H.264) format as this is the mostpopular among personal media devices. However, alternatives are alsouseable and a transcoding mechanism can be provided in one or moreembodiments. Such a transcoding mechanism can include, for example, anadjunct digital signal processor (DSP) within the STB 901 or as afunction of a general purpose processor with sufficient computationalspeed for a software implementation. A variety of hardware and softwaretranscoders are available and known to the skilled artisan who, giventhe teachings herein, will be able to adapt same for use with one ormore embodiments of the invention. Content can be added to the contentsource via Internet connection, cable service (DVR), or an externalstorage device 205. Content can also be streamed directly from anexternal service provider (e.g., from head end 150). Content source 901preferably has Internet connectivity via radio frequency (RF) 904 or aseparate wireless router (not shown). Element 905 depicts intranetconnectivity.

In some instances, a STB could be designed that entirely relies on link905 and communicates with both Internet and intranet. The Internet wouldbe accessed via a wireless router in the home. Both scenarios arecontemplated as in block 204, which depicts a network connection that isconnected to a home network via wireless or wired connection to awireless router. A router is sometimes referred to as a gateway becauseit is what connects the home (intranet) to the public Internet. Thus,instead of direct RF connectivity to the Internet, such as via a coaxialcable 904, unit 901 could have a wired or wireless 905 link to a routerwhich itself connected to the Internet.

Processor 201 extracts the composite video and multiple-audio data fromthe storage device 203, 205 and buffers it in memory for timelydistribution to a graphics processor 207. In other instances, streamingcan be employed—for example, the content source 901 tunes (for examplewith a QAM RF tuner) to a television channel which has, say, an Englishsoundtrack with a Spanish simulcast. A QAM demodulator pulls in the datastream and then the process is similar to the case when a program isaccessed from the disk drive, except that it is received in a serialfashion. Thus, in this (streaming) instance, the composite video andmultiple audio data is, in essence, extracted from a quadratureamplitude modulated RF signal or equivalent. Non-limiting examples ofequivalents include data over fiber optics or an IP stream in H.264 orthe like obtained over a wired or wireless network.

The movement from memory to the graphics processor may include avariable delay mechanism 208 (data FIFO or other) such that if there isa propagation delay between the local audio and display and thehand-held audio devices, the local audio/video display may be held backso that the display and audio may be synchronized correctly with theaudio channel being provided to the external portable media devices orother alternative text-receiving devices. The graphics processor 207converts the digital video signal into one appropriate for the displaydevice; for example, analog composite, component, or digital via HDMI®(High Definition Multimedia Interface; registered mark of HDMILICENSING, L.L.C., 1060 EAST ARQUES AVENUE, SUITE 100, SUNNYVALE Calif.94085 USA) or Digital Visual Interface (DVI). Audio is sent to the audioprocessor 206 via the same delay mechanism for local audio playback. Thedelay for both audio and video will be the same in order to maintainsynchronization. One or more embodiments additionally distributealternative audio content in the form of audio data (MP3 and the like)directly to the network interface 204 for distribution to a PMD 903. Thepreferred mode of transmission is that of non-compressed pulse codemodulated audio wrapped in IP frames. The IP frames should be short induration in order to minimize the requirement for buffering and henceadditional delay that would have to be compensated for in order tomaintain synchronization between the remote audio devices and the localaudio and video. “Short” in this context means short with respect to thebuffer size. In a non-limiting example, a 100 byte packet would induce a2.3 millisecond delay. Other instances could have different values. Ingeneral, the packets should be sufficiently short such that they do notintroduce so much delay such that synchronization between the sound andthe video track becomes an issue.

User interface 401, 504 is activated when the application is executed onthe PMD. The application uses the DLNA stack to contact the contentsource 901 and obtain descriptive information concerning the alternativeaudio and text content available. Alternatively, the software could setup a stream of video and audio data from an external service providervia the Internet or other mechanism (e.g., MPEG over quadratureamplitude modulation (QAM) such as in network 101 and the like). Usersbegin playing the video and local audio through the local display 902and the user with the personal media device selects the alternativeaudio or text content desired. If audio is selected, the personal mediadevice then requests the content source 901 to stream the appropriateaudio to it. Once received via the IP stack 404 and network interfacecontroller (NIC) 505, the application sends the data to the audio driver402 which interfaces to the local audio hardware 503 and provides soundto the listener. If text is selected, the user interface 401 displaysthe text in a synchronous manner with the display of the video contentvia the graphics processor 504. Synchronization is maintained betweenthe PMD audio, PMD text, and the video presentation via limiting thepropagation delay between the audio packets sent to the decoder for theexternal audio devices and the audio packets sent to the RF channel thatconnects to the personal media devices. It should be noted thatsynchronization is believed to be less of an issue in the case ofsubtitles.

Furthermore in this regard, there is, in at least some instances, apropagation delay in transmitting alternative audio or subtitles over tothe personal media device. Thus, a variable delay buffer 208 can beemployed going to the TV decoder 207, 206 and adjustments made so thatby time the audio and video show up to the TV the correspondingalternative audio or subtitles also show up to the decoder (Graphics andAudio Processors 207, 206 which decode the digital data into the outputsignals.), and, through the decoder, within the personal media device.Programmable variable delay 208 can be implemented, for example, with aFIFO buffer.

Thus, the system preferably also provides an adjustment parameter to adda small propagation delay between the video and audio being delivered tothe display and main audio and the packets sent to the PMDs, as shown at208.

It will be appreciated that one or more embodiments allow people whospeak different languages (e.g., multiple generations of an immigrantfamily) to view content in the same room on the same TV, withaccommodation of different language support in a simultaneous viewingscenario. One or more instances provide multiple audio streamssimultaneously along with any given video content such that audio indifferent languages may be received simultaneously with the video andeach other by multiple receiver/decoder devices that allow the selectionand playback of one of the audio streams provided.

In one or more embodiments, an entity such as a cable multiple systemoperator (MSO) facilitates download, installation, and launch of asuitable audio reception, selection and playback application (see, e.g.,FIG. 5) by the customer onto his or her PMD. Upon launching theapplication, the user is prompted for his or her language preferenceamong the audio available within the content. Once selected, the PMDconnects to the proper audio stream for that particular language (oreven an enhanced “director's commentary” audio program). The PMDsoftware and hardware combination then receives, decodes, and presentsthe audio to the user connected via that particular PMD. Multicastsynchronization is preferably implemented to ensure that the video andall audio streams are in sync with each other. A variety of currenttechnologies are available for such synchronization. Non-limitingexamples are disclosed in US Patent Publications 2008-0152309 and2005-0276282, and in U.S. Pat. Nos. 5,594,660 and 5,959,684. Thecomplete disclosures of US Patent Publications 2008-0152309 and2005-0276282, and U.S. Pat. Nos. 5,594,660 and 5,959,684 are expresslyincorporated herein by reference in their entireties for all purposes.

One or more embodiments thus provide “video stream bonding” whereinmultiple portable audio/video devices, e.g. IPOD®, IPHONE®, IPAD®(registered marks of Apple Inc., Cupertino, Calif., USA); Zune®(registered mark of Microsoft Corporation, Redmond, Wash., USA);ANDROID® Phone, and the like are synchronized to the same video stream,with each presenting its own synchronized audio, closed captioning, andprivate data stream. One or more embodiments also enablemulti-linguistic advertising, as in the case of different commercialvideo targeted to a given device's language and/or cultural settings, aswell as interactive applications.

Some embodiments can be employed to stream multiple audio channels alongwith the video from existing high end television set top boxes, withsuitable modifications to the set top boxes and PMDs, as describedherein.

One or more embodiments provide support for same-room video viewing inmulti-linguistic households, for the hearing impaired, and in cases ofsimilar language or audio barriers. One or more embodiments allowcustomers with bi-lingual family members to view content together intheir preferred or native language. In some instances, multiple-languagesubtitles are offered for those with both linguistic difference andaural impairment.

One or more instances, rather than amplifying audio for hearing impairedindividuals, further allow one to watch video content in a preferredlanguage without disturbing others. One or more embodiments allowcontent distributors to target and provide culturally preferredadvertising and similar data. In another aspect, personalized closedcaptioning can be provided in some cases, as well as subtitles that areselectable on a language or enhanced feature basis.

In a non-limiting exemplary use case, a group may watch a video ondevice 902 while one or more members listen to the audio in a primarylanguage; for example, on a speaker system associated with device 902.One or more other users listen in one or more different language viaheadphones on one or more corresponding personal media devices; thesoundtracks in the alternate languages are synchronized with the videoas is the audio in the primary language.

It is worthwhile noting that some embodiments can make use of anintegrated circuit chip such as the Cirrus CF 5368 chip which can decodeup to eight channels of audio. In a non-limiting example, such chip or asimilar chip can be located on the PMD to pull out the audio packets.The Cirrus CF 5368 chip can be used, for example, in conjunction withthe Cirrus CS4384 SPDIF receiver.

A variety of techniques can be used to communicate data between the PMDs903 and device 901; e.g., Wi-Fi, IEEE 802.11, or other radio frequency(RF) mechanism, infrared (IR), Multimedia over Coax Alliance (MoCA), andthe like. In some instances, the device 901 may send SAP signals (SIMAccess Profile, a profile within BLUETOOTH) to the PMDs 903. The videoand multiple audio data may, for example, be in MPEG-2 or MPEG-4. Insome instances, one or more embodiments make use of S/PDIF, which is adata link layer protocol and a set of physical layer specifications forcarrying digital audio signals between devices and components overeither optical or electrical cable. S/PDIF is standardized in IEC 60958as IEC 60958 type II. In such cases, the secondary audio may beextracted from the source and put into an S/PDIF stream, which is thenencapsulated into TCP/IP (transfer control protocol/internet protocol)or other RF envelope for transmission to the PMDs 903. Software in thePMDs 903 pulls the audio out of the IP encapsulation, or a suitable chipis used to extract the audio packets, such as the above-mentioned CirrusCF 5368 chip. In one or more embodiments, the S/PDIF connects to the CS5368.

Note that in some instances, device 901 could also be realized as aWiFi- or MoCA-enabled DVD or BLURAY player interfacing with the PMDs 903through a router.

By way of review, in FIG. 3, elements 201, 202, 203, and 205 arewell-known physical devices or elements. Programmable variable delay 208can be implemented, for example, by a FIFO buffer or the like which is aphysical device. Audio and graphics processors 206 and 207 can beimplemented, for example, as hardware elements, in a well-known manner.Audio processor 206 can be implemented as a digital-to-analog converter,for example. Graphics processor 207 can be implemented as a video boardwith one or more high-speed specialized chips, for example. Block 204represents appropriate physical connectors and/or a transceiver andassociated circuitry, for example. The elements can be connected by aphysical data bus or the like.

Still continuing the review, in FIG. 4, block 301 represents datapatterns extracted from a DVD or hard drive or obtained via streamingand then loaded into a physical memory; block 302 represents awell-known series of software components (Digital Living NetworkAlliance (DLNA)) that are stored on, e.g., disk drive 203, loaded intomemory 202 and executed on processor 201 to cause the processor to carryout a series of steps; and block 303 represents the well-known internetprotocol stack which can be obtained on a physical chip or, depending onthe device, as a software stack stored, loaded, and executed in asimilar manner.

Again, still continuing the review, in FIG. 5, block 402 represents asmall, low-level piece of software which sets the registers within aphysical digital-to-analog converter for purposes of volume changes andthe like; block 403 represents a well-known series of softwarecomponents (Digital Living Network Alliance (DLNA)) that are stored on,e.g., on non-volatile storage on the PMD 903, loaded into memory 502 andexecuted on processor 501 to cause the processor to carry out a seriesof steps; and block 404 represents the well-known internet protocolstack which can be obtained on a physical chip or, depending on thedevice, as a software stack stored, loaded, and executed in a similarmanner. Block 401 represents an application which can, for example, bestored, loaded, and executed in a similar manner, and which includesinstructions which cause the processor, in conjunction with blocks 504and 506, to display prompts which allow the user to select the desiredsecondary audio asset (for example, simply a button which continuouslychanges the language that is decoded or a pair of buttons for movingforwards and backwards through the selections).

Yet further, by way of review, in FIG. 6, elements 501, 502, and 506 arewell-known physical devices or elements. Audio and graphics processors503 and 504 can be implemented, for example, as hardware elements, in awell-known manner. Audio processor 503 can be implemented as adigital-to-analog converter, for example. Graphics processor 504 can beimplemented as a video board with one or more high-speed specializedchips, for example, or a simpler display approach can be used in a PMDin some instances. Block 505 represents appropriate physical connectorsand/or a transceiver and associated circuitry, for example. The elementscan be connected by a physical data bus or the like. Other input devicesbesides or in addition to keyboards can be employed.

Reference should now be had to flow chart 600 of FIG. 7, which begins instep 602. Given the discussion thus far, it will be appreciated that, ingeneral terms, an exemplary method, according to an aspect of theinvention, includes the step 608 of extracting digital video data anddigital multiple-audio data from a source (e.g., 203, 205, or streamingfrom a network such as 101), using a hardware processor 201 in a contentsource device 901 within a premises. An additional step 610 includesprocessing the extracted digital video data for display on a maindisplay device (e.g., 902) in the premises (for example, making use ofgraphics processor 207 as discussed above). A further step 612 includesprocessing the extracted digital multiple-audio data into a primarysoundtrack in a primary language, to be listened to within the premisesin synchronization with the displayed extracted digital video data (forexample, making use of audio processor 206 as discussed above). Theprimary soundtrack corresponds to the displayed extracted digital videodata, in the primary language. For example, the primary soundtrack couldbe English played over speakers associated with display 902, andsynchronized with the video thereon. As used herein, a “soundtrack”simply means audio associated with a video, and may include speechand/or music, for example. A further step 618 includes processing theextracted digital multiple-audio data into at least one secondary audioasset, different than the primary soundtrack. An MPEG stream is a seriesof data including tables (program map tables) which describe theprograms, which in turn include different audio and video streams. Thedescription of the audio streams typically includes the three-characterISO 639 language descriptor.

As used herein, a “secondary audio asset” includes:

-   -   a secondary soundtrack in a different language than the first        soundtrack,    -   a secondary soundtrack in the same language as the first        soundtrack, but different from the first soundtrack (for        example, descriptive audio in the same language as the first        soundtrack), and/or    -   captions which textually represent audio, in the same or a        different language as a primary or secondary soundtrack.

The soundtrack and/or text may pertain to the “main” video, e.g., amovie, and/or to alternative video features associated with the “main”video (e.g., ancillary information such as a director's commentary orthe like).

A further step 620 includes transmitting the at least one secondaryaudio asset to a personal media device 903 within the premises (forexample, via block 204), for apprehension by a user of the personalmedia device in synchronization with the displayed extracted digitalvideo data (i.e., soundtrack or captions are aligned with the video). Ina preferred but non-limiting approach, such synchronization is based onmanual feedback from the user in the case of a secondary soundtrack, andis inherent in the case of captions due to the fact that typicalpropagation delays simply do not significantly impact the apprehensionof textual captions in connection with the video.

Processing Continues at 622.

With respect to multiple secondary audio assets, a number of approachesare possible (for example, primary soundtrack is English and secondarysoundtracks in both Spanish and French). In some cases, an 8-channeldevice is employed with two stereo channels per language allowing fourlanguages on a single chip (e.g., Cirrus CF 5368). It is believed thatonce any necessary delay has been introduced in the video display andprimary soundtrack, all of the secondary audio assets will be adequatelysynchronized as they will likely arrive at the PMDs at the same time orsufficiently close to each other such that the synchronization will beadequate.

In a preferred approach, step 620 includes transmitting as packetsaddressed to the personal media device. In instances where UniversalPlug and Play (UPnP) is employed, UPnP will provide the appropriateaddresses.

As noted, in some cases, the at least one secondary audio assettransmitted in the transmitting step is a secondary soundtrack in asecondary language and/or a descriptive audio track. The secondarysoundtrack corresponds to the displayed extracted digital video data, inthe secondary language. In such cases, the transmitting step 620 caninclude, by way of example and not limitation, transmittingnon-compressed pulse code modulated audio wrapped in IP frames. Captionscould be transmitted, for example, encapsulated in IP frames. Forexample, in addition to Program and System Information Protocol (PSIP)data with the actual audio, text can easily be passed using wireless IPor the like. In a non-limiting example, the captions are in accordancewith ISO/IEC 13818-3:1998(E) Appendix E Program related text (closedcaptioning).

Examples of the source include a persistent memory device such as 203coupled to the hardware processor 201; a media drive 205, coupled to thehardware processor 201, and having a removable media therein; andcontent streamed to the content source device from an external serviceprovider (e.g., over the Internet or via a video content network such asHFC network 101, a FTTH/FTTC network, a satellite network, or the like).

As noted, instead of or in addition to actual audio which can be playedover a headphone or the like, in some cases, the secondary audio assettransmitted in the transmitting step 620 includes subtitles for displayon the personal media device. It is currently contemplated that thesesubtitles are displayed by themselves, i.e., not superimposed overanother display of the video on the PMD, because of the large amount ofbandwidth needed to pass video. However, as higher bandwidth homenetworks become more commonplace, in some instances, display of video onthe PMD is also possible. Such subtitles could be in the same languageas the primary language, so that someone with a hearing disability butwho speaks the same language as others can view them without disturbingothers by their presence on the main display 902. In other cases, thesubtitles transmitted for display on the personal media device aresubtitles, in a language other than the primary language. In eithercase, the transmitted subtitles correspond to the displayed extracteddigital video data.

Note that use of the term “subtitles” is not intended to be limiting asto where on the display the text is displayed to the user.

Optional step 606 includes obtaining, at the content source device 901,a menu selection from the user of the personal media device 903 (forexample wirelessly via block 204). The menu selection specifies a chosenform of the second audio asset (i.e., what language, whether audio orcaptions or both, and so on). This step is optional—as noted, in somecases, all the available secondary audio assets are transmitted and aselection is made at the PMD using block 401.

Thus, again note that in one or more alternative embodiments, alllanguages available are transmitted from unit 901 and the PMD makes theselection. The interface could be as simple as selecting one of fouraudio channels to see what is available or providing a secondary IP pathto provide a description of what audio is on what audio channel. Forexample, simply encapsulate the output of the PSIP in RF and send it toa receiver on the PMD where a selection is made as to which of, say,four stereo streams is to be played back—in the simplest case, noindication of the language need even be given to the user, who simplylistens to all (say, four) available channels until finding the desiredlanguage. All languages may be multiplexed together, for example, andthe user obtains the desired one by having the PMD identify the packetsin that language by their headers.

In other instances, provide a filtering function wherein only thesecondary audio assets requested by users of PMDs are actual sent fromsource 901 to PMD 903, advantageously saving bandwidth in the homenetwork.

Optional step 604 includes providing, to the personal media device 903,from the content source device 901 a representation of a menu from whichthe menu selection can be made. The representation of the menu includesdata indicative of available forms of the second audio asset. The menucan be displayed, for example, via blocks 401, 504, with menu selection,for example, via block 506 or a similar technique such as a pointingdevice.

Optional step 618 includes introducing a delay in the extracted digitalvideo data for display on the main display device 902 and in the primarysoundtrack in the primary language (e.g., as sent to processor 206), toaccount for the delay in the transmitting of the secondary audio assetto the personal media device. Delay can be introduced, for example, viablock 208. The delay is introduced to maintain the synchronization ofthe secondary audio asset with the displayed extracted digital videodata. Issues with the synchronization can be detected, for example, indecision block 614. If no problem (“YES” branch indicating delay is OKas-is), bypass step 616; if there is a problem (“NO” branch indicatingdelay is problematic), make required adjustments in step 616.

In some instances, the delay in the FIFO is set via feedback from thePMD user. For example, an IP channel is provided with control signalswhich pass from the PMD back to source 901 to lengthen or shorten thedelay; for example, by lengthening or shortening the ring bufferemployed for the FIFO to increase or reduce the delay, respectively.Other embodiments could employ automated techniques as set forth, forexample, in the above-mentioned US Patent Publications 2008-0152309 and2005-0276282, and U.S. Pat. Nos. 5,594,660 and 5,959,684.

In some embodiments, language-specific ancillary information, such asthe above-mentioned director's commentary, is displayed simultaneouslyto different viewers in different languages, unlike current DVD metadatawherein only one language at a time is possible.

Reference should now be had to flow chart 700 of FIG. 8, which begins instep 702, and depicts a method for interaction between a personal mediadevice 903 and a content source device 901 which provides, within apremises, digital video data for display on a main display device 902 inthe premises, and a primary soundtrack in a primary language, to belistened to within the premises in synchronization with the displayeddigital video data. One step 706 includes obtaining, at the personalmedia device within the premises, from the content source device withinthe premises, a representation of a menu. This can be done, for example,via blocks 204, 505. The representation of the menu includes dataindicative of available forms of a secondary audio asset, as discussedabove. An additional step 708 includes providing, from the personalmedia device, to the content source device, a menu selection specifyinga chosen form of the secondary audio asset. A further step 710 includesobtaining, at the personal media device, from the content source device,the secondary audio asset, for apprehension by a user of the personalmedia device in synchronization with the displayed digital video data.The secondary audio asset is different than the primary soundtrack (asdiscussed above, textual material such as comprises subtitles fordisplay on the personal media device and/or a secondary soundtrack in adifferent (secondary) language that corresponds to the displayed digitalvideo data). As noted, such subtitles could be in the same language asthe primary language, so that someone with a hearing disability but whospeaks the same language as others can view them without disturbingothers by their presence on the main display 902. In other cases, thesubtitles transmitted for display on the personal media device aresubtitles, in a language other than the primary language. In eithercase, the transmitted subtitles correspond to the displayed extracteddigital video data.

Processing Continues at Step 716.

It is to be emphasized that FIG. 8 is a non-limiting example, and insome cases, as noted above, all the available secondary audio assets aretransmitted and a selection is made at the PMD using block 401.

As noted, in some cases, step 710 includes obtaining the secondary audioasset as packets specifically addressed to the personal media device.Again, in some instances, all the secondary audio data is multiplexedtogether and the PMD determines which of the data is to be decoded; instill other instances, different frequencies or the like are availableand the PMD tunes to the desired frequency.

As also noted, step 710 can be carried out in a variety of ways; whenthe secondary audio asset is a secondary soundtrack or a descriptiveaudio track, one non-limiting method includes obtaining non-compressedpulse code modulated audio wrapped in IP frames.

Optionally, an additional step 714 includes providing, from the personalmedia device, to the content source device, data indicative of anappropriate delay to be introduced in the displayed digital video dataand the primary soundtrack so as to account for any delay in theobtaining, at the personal media device, from the content source device,the secondary audio asset. Such data is provided (for example, viablocks 204, 505) to maintain the synchronization of the secondary audioasset with the displayed digital video data, by adjusting variable delay208. As used herein, “data indicative of an appropriate delay” includesdata to set a delay where none exists and/or data to adjust or eliminatean existing delay. Refer to decision block 712, wherein step 714 is onlycarried out in case the current delay (if any) is not acceptable, as perthe “NO” branch; otherwise, step 714 is bypassed as per the “YES” branchwherein the existing delay (if any) is acceptable.

Optional step 704 includes, for example, the case wherein the contentsource device 901 is provided by (i.e., to the consumer; it may well bemanufactured by someone else) a multi-services operator; themulti-services operator invites the user of the personal media device todownload thereto an appropriate software application (e.g., part or allof components in FIG. 5) which enables the personal media device atleast to provide the menu selection to the content source device (theapplication might be downloadable directly from the MSO or the MSO mightmake it available to an “app store” from which the user downloads it).In other instances, the application could be provided to the PMD inanother fashion.

At this point, it should be noted that any of the methods depictedherein can also include the actual display of video on the main screen902 and/or display of the PMD; the actual display of captions on themain screen 902 and/or display of the PMD, and the actual reproductionof sounds corresponding to the soundtrack(s) by, for example, headphoneson the PMD and/or speakers associated with the main screen 902.

In some instances, all soundtracks may be played on PMD headphones withnone played out loud on speakers associated with the main screen 902.

In another aspect, an exemplary system can include device 901, device903, or both device 901 and 903.

Thus, for example, an exemplary system includes a main memory 202; amain hardware processor 201 coupled to the main memory; a source (e.g.,203, 205, network 101 or Internet) of digital video data and digitalmultiple-audio data (e.g., a main soundtrack and at least one secondaryaudio asset) in data communication with the main hardware processor; awireless interface 204 coupled to the main hardware processor; and agraphics processor 207 coupled to the main hardware processor. As usedherein, a “wireless interface” is intended to encompass wirelesshardware in the device 901 or a wired connection configured forconnection to a wireless router.

The main hardware processor 201 and the main memory 202 arecooperatively configured to extract the digital video data and thedigital multiple-audio data from the source; process the extracteddigital video data and route the extracted video data to the graphicsprocessor 207 for display in a premises; and process the extracteddigital multiple-audio data into a primary soundtrack in a primarylanguage, to be listened to within the premises in synchronization withthe displayed extracted digital video data. The primary soundtrackcorresponds to the displayed extracted digital video data, in theprimary language. The main hardware processor 201 and the main memory202 are also cooperatively configured to process the extracted digitalmultiple-audio data into a secondary audio asset (as discussed above),different than the primary soundtrack (i.e., text in the same ordifferent language or audio in a different language). The wirelessinterface 204 is configured to facilitate transmitting the secondaryaudio asset to a personal media device 903 within the premises, forapprehension by a user of the personal media device in synchronizationwith the displayed extracted digital video data.

As noted, wireless interface 204 is preferably configured to facilitatetransmitting the secondary audio asset as packets addressed to thepersonal media device 903.

The system preferably also includes an audio processor 206 whichfacilitates presentation of the primary soundtrack. In some instances,wireless interface 204 includes a wireless transmitter (including, ofcourse, a suitable antenna or the like) which transmits the secondarysoundtrack and/or descriptive audio track as non-compressed pulse codemodulated audio wrapped in IP frames. Captions could be provided viaanother IP stream, for example.

In a preferred embodiments, the system also includes a variable delayblock 208 upstream of the graphics processor 207, which introduces adelay in the extracted digital video data that is routed to the graphicsprocessor, to account for a delay in the transmitting of the secondaryaudio asset to the personal media device, to maintain thesynchronization of the secondary audio asset with the displayedextracted digital video data. In a general case, both a delay and aspeeding up are provided for.

In some instances, the main hardware processor 201, the main memory 202,and the wireless interface 204 are cooperatively configured to provideto the personal media device a representation of a menu from which amenu selection can be made. The representation of the menu includes dataindicative of available forms of the second audio asset. The mainhardware processor 201, the main memory 202, and the wireless interface204 are also cooperatively configured to obtain the menu selection fromthe user of the personal media device. The menu selection specifies achosen form of the second audio asset. Once again is to be emphasizedthat this is a non-limiting example, and in some cases, as noted above,all the available secondary audio assets are transmitted and a selectionis made at the PMD using block 401.

As noted, an exemplary system can include device 901, device 903, orboth device 901 and 903. Thus, in some cases, the system just describedalso includes the personal media device 903, which in turn includes apersonal media device hardware processor 501; a personal media devicewireless interface 505 coupled to the personal media device hardwareprocessor; an input-output interface coupled to the personal mediadevice hardware processor; and a personal media device memory 502 whichis coupled to the personal media device hardware processor and whichstores, in a non-transitory manner, a software application which, whenexecuted by the personal media device hardware processor, causes theinput-output interface to present to the user the representation of themenu and obtain from the user the menu selection. The personal mediadevice wireless interface 505 transmits the menu selection to thewireless interface 204 coupled to the main hardware processor 201.

In some instances, the software application stored in the personal mediadevice memory 502, when executed by the personal media device hardwareprocessor 501, further causes transmission, by the personal media devicewireless interface 505, to the wireless interface 204 coupled to themain hardware processor 201, of data indicative of an appropriate delay(as defined above) to be introduced in the digital video data and theprimary soundtrack so as to account for a delay in the obtaining, at thepersonal media device, from the content source device, of the secondaryaudio asset. The delay is introduced to maintain the synchronization ofthe secondary audio asset with the displayed extracted digital videodata.

One or more additional aspects include an overall system within thecontext of a video content network such as that depicted in FIGS. 1, 1A,and 1B; the individual PMD with the appropriate application thereon, asin FIGS. 5 and 6; computer program products embodied in a tangible,computer-readable, recordable storage medium, such as, for example, allor part of the software-implemented blocks in FIGS. 4 and 5; means forcarrying out the method steps, and the like.

System and Article of Manufacture Details

The invention can employ hardware aspects or a combination of hardwareand software aspects. Software includes but is not limited to firmware,resident software, microcode, etc. One or more embodiments of theinvention or elements thereof can be implemented in the form of anarticle of manufacture including a machine readable medium that containsone or more programs which when executed implement such step(s); that isto say, a computer program product including a tangible computerreadable recordable storage medium (or multiple such media) withcomputer usable program code configured to implement the method stepsindicated, when run on one or more processors. Furthermore, one or moreembodiments of the invention or elements thereof can be implemented inthe form of an apparatus including a memory and at least one processorthat is coupled to the memory and operative to perform, or facilitateperformance of, exemplary method steps.

Yet further, in another aspect, one or more embodiments of the inventionor elements thereof can be implemented in the form of means for carryingout one or more of the method steps described herein; the means caninclude (i) hardware module(s), (ii) software module(s) executing on oneor more hardware processors, or (iii) a combination of hardware andsoftware modules; any of (i)-(iii) implement the specific techniques setforth herein, and the software modules are stored in a tangiblecomputer-readable recordable storage medium (or multiple such media).Appropriate interconnections via bus, network, and the like can also beincluded.

FIG. 9 is a block diagram of a system 2100, processor 2120 of which isrepresentative of processors associated with servers, clients, set topterminals or other devices 901, DBWAD, SRM, GSRM, MAS 104A, personalmedia devices, and any other elements with processing capabilitydepicted in the other figures. In one or more embodiments, inventivesteps are carried out by one or more of the processors in conjunctionwith one or more interconnecting network(s).

As shown in FIG. 9, memory 2130 configures the processor 2120 toimplement one or more aspects of appropriate methods, steps, andfunctions (collectively, shown as process 2180 in FIG. 9). The memory2130 could be distributed or local and the processor 2120 could bedistributed or singular. The memory 2130 could be implemented as anelectrical, magnetic or optical memory, or any combination of these orother types of storage devices. It should be noted that if distributedprocessors are employed, each distributed processor that makes upprocessor 2120 generally contains its own addressable memory space. Itshould also be noted that some or all of computer system 2100 can beincorporated into an application-specific or general-use integratedcircuit. For example, one or more method steps could be implemented inhardware in an ASIC rather than using firmware. Display 2140 isrepresentative of a variety of possible input/output devices (e.g.,mice, keyboards, printers, etc.).

As is known in the art, part or all of one or more aspects of themethods and apparatus discussed herein may be distributed as an articleof manufacture that itself includes a computer readable medium havingcomputer readable code means embodied thereon. The computer readableprogram code means is operable, in conjunction with a computer systemincluding hardware, to carry out all or some of the steps to perform themethods or create the apparatuses discussed herein. The computerreadable medium may be a recordable medium (e.g., floppy disks, harddrives, compact disks, EEPROMs, or memory cards) or may be atransmission medium (e.g., a network including fiber-optics, theworld-wide web, cables, or a wireless channel using time-divisionmultiple access, code-division multiple access, or other radio-frequencychannel). Any medium known or developed that can store informationsuitable for use with a computer system may be used. Thecomputer-readable code means is any mechanism for allowing a computer toread instructions and data, such as magnetic variations on a magneticmedium or height variations on the surface of a compact disk. As usedherein, a tangible computer-readable recordable storage medium isintended to encompass a recordable medium which stores instructionsand/or data in a non-transitory manner, examples of which are set forthabove, but is not intended to encompass a transmission medium ordisembodied signal.

The computer systems and servers described herein each contain a memorythat will configure associated processors to implement the methods,steps, and functions disclosed herein. Such methods, steps, andfunctions can be carried out, e.g., by processing capability onindividual elements in the other figures, or by any combination thereof.The memories could be distributed or local and the processors could bedistributed or singular. The memories could be implemented as anelectrical, magnetic or optical memory, or any combination of these orother types of storage devices. Moreover, the term “memory” should beconstrued broadly enough to encompass any information able to be readfrom or written to an address in the addressable space accessed by anassociated processor. With this definition, information on a network isstill within a memory because the associated processor can retrieve theinformation from the network.

Thus, elements of one or more embodiments of the present invention canmake use of computer technology with appropriate instructions toimplement method steps described herein. For example, a master copy 2199of the PMD application of FIG. 5 may be stored in association with an“app store” or the like and downloaded to the PMD 903 over a wired orwireless network, as seen in FIG. 9.

As used herein, including the claims, a “server” includes a physicaldata processing system (for example, system 2100 as shown in FIG. 9)running a server program. It will be understood that such a physicalserver may or may not include a display, keyboard, or other input/outputcomponents.

Furthermore, it should be noted that any of the methods described hereincan include an additional step of providing a system comprising distinctsoftware modules embodied on one or more tangible computer readablestorage media. All the modules (or any subset thereof) can be on thesame medium, or each can be on a different medium, for example. Themodules can include any or all of the components shown in the figures(e.g. modules/submodules in FIGS. 4 and 5, DBWAD, SRM/GSRM, MAS, and soon). The method steps can then be carried out using the distinctsoftware modules of the system, as described above, executing on the oneor more hardware processors. Further, a computer program product caninclude a tangible computer-readable recordable storage medium with codeadapted to be executed to carry out one or more method steps describedherein, including the provision of the system with the distinct softwaremodules. In one or more embodiments, the code is stored in anon-transitory manner.

Non-limiting examples of languages that may be used include markuplanguages (e.g., hypertext markup language (HTML), extensible markuplanguage (XML), standard generalized markup language (SGML), and thelike), C/C++, assembly language, Pascal, Java, EBIF—Extended BinaryInterchange Format language, UNIX shell scripts (for example, togenerate information to supply to the GSRM), Objective C (used, forexample, in certain PMDs from Apple Inc.) and the like. Note that EBIFwould typically only be employed in connection with a set-top box. RTSPand/or RPC can be employed for interface protocols, for example.Furthermore, non-limiting examples of useful database software includeAccess® software (registered mark of Microsoft Corporation, Redmond,Wash., USA); Oracle® software (registered mark of Oracle InternationalCorporation, 500 Oracle Parkway, Redwood City, Calif. 94065, USA);Informix® software (registered mark of International Business MachinesCorporation, Armonk, N.Y., USA); and structured query language (SQL)software available from many sources, including Microsoft Corporation,Redmond, Wash., USA).

Accordingly, it will be appreciated that one or more embodiments of theinvention can include a computer program including computer program codemeans adapted to perform one or all of the steps of any methods orclaims set forth herein when such program is implemented on a processor,and that such program may be embodied on a tangible computer readablerecordable storage medium. Further, one or more embodiments of thepresent invention can include a processor including code adapted tocause the processor to carry out one or more steps of methods or claimsset forth herein, together with one or more apparatus elements orfeatures as depicted and described herein.

System(s) have been described herein in a form in which variousfunctions are performed by discrete functional blocks. However, any oneor more of these functions could equally well be embodied in anarrangement in which the functions of any one or more of those blocks orindeed, all of the functions thereof, are realized, for example, by oneor more appropriately programmed processors such as digital signalprocessors (DSPs). A DSP typically comprises a combination of digitallogic devices and other components, which may be a state machine orimplemented with a dedicated microprocessor or micro-controller runninga software program or having functions programmed in firmware.

Although illustrative embodiments of the present invention have beendescribed herein with reference to the accompanying drawings, it is tobe understood that the invention is not limited to those preciseembodiments, and that various other changes and modifications may bemade by one skilled in the art without departing from the scope orspirit of the invention.

1. (canceled)
 2. A method comprising the steps of: processing, by acontent source device within a premises, digital video data for displayon a main display device in said premises; processing, by said contentsource device, digital multiple-audio data into a primary soundtrack ina primary language, to be listened to within said premises insynchronization with said digital video data displayed by said maindisplay device, said primary soundtrack corresponding to said displayeddigital video data, in said primary language; introducing a delay insaid digital video data for display on said main display device and insaid primary soundtrack in said primary language, wherein said delay iscontrolled by said content source device using feedback received by saidcontent source device from a personal media device; processing saiddigital multiple-audio data into at least one secondary audio asset,different than said primary soundtrack; and transmitting said at leastone secondary audio asset to said personal media device within saidpremises, wherein said delay of said displayed digital video data andsaid primary soundtrack is synchronized to a playback of said at leastone secondary audio asset by said personal media device.
 3. The methodof claim 2, wherein said transmitting comprises transmitting as packetsaddressed to said personal media device.
 4. The method of claim 2,wherein transmitting said at least one secondary audio asset comprisestransmitting non-compressed pulse code modulated audio wrapped in IPframes.
 5. The method of claim 2, wherein said at least one secondaryaudio asset transmitted in said transmitting step comprises at least oneof: a secondary soundtrack in a secondary language, said secondarysoundtrack corresponding to said displayed digital video data, in saidsecondary language; and a descriptive audio track.
 6. The method ofclaim 2, wherein said at least one secondary audio asset transmitted insaid transmitting step comprises subtitles for display on said personalmedia device.
 7. The method of claim 6, wherein in said transmittingstep, said subtitles transmitted for display on said personal mediadevice comprise subtitles, in a language other than said primarylanguage, corresponding to said displayed digital video data.
 8. Themethod of claim 2, further comprising obtaining, at said content sourcedevice, a menu selection from said personal media device, said menuselection specifying a chosen form of said second audio asset.
 9. Themethod of claim 8, further comprising providing, to said personal mediadevice, from said content source device, a representation of a menu fromwhich said menu selection can be made, said representation of said menucomprising data indicative of available forms of said second audioasset.
 10. A method comprising the steps of: obtaining, at a personalmedia device in a premises, from a content source device in saidpremises, a secondary audio asset; and providing, from said personalmedia device to said content source device, data indicative of a delaycausing said content source device to delay display of digital videodata and playback of a primary soundtrack synchronizing a playback ofsaid at least one secondary audio asset by said personal media devicewith said display of said digital video data and said playback of saidprimary soundtrack.
 11. The method of claim 10, wherein said obtainingof said secondary audio asset comprises obtaining as packetsspecifically addressed to said personal media device.
 12. The method ofclaim 10, wherein said secondary audio asset comprises at least one of:a secondary soundtrack in a secondary language, said secondarysoundtrack corresponding to said displayed digital video data, in saidsecondary language; and a descriptive audio track.
 13. The method ofclaim 12, wherein said obtaining of said secondary audio asset comprisesobtaining non-compressed pulse code modulated audio wrapped in IPframes.
 14. The method of claim 10, wherein said secondary audio assetcomprises subtitles for display on said personal media device.
 15. Themethod of claim 14, wherein in said step of obtaining said secondaryaudio asset, said subtitles comprise subtitles in a language other thansaid primary language, corresponding to said displayed digital videodata.
 16. The method of claim 10, wherein said content source device isprovided by a multi-services operator, further comprising saidmulti-services operator inviting a user of said personal media device todownload thereto an appropriate software application which enables saidpersonal media device at least to provide a menu selection to saidcontent source device specifying a chosen form of said secondary audioasset.